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ABSTRACT 

Infrared  (IR)  images  in  the  3  to  5  and  8  to  12  micron  band  were  taken  of  soldiers  wearing 
various  camouflaged  uniforms.  The  soldiers  wearing  the  uniforms  were  either  standing, 
crouched  or  prone.  The  images  were  presented  to  20  observers  in  the  TARDEC  Visuat 
Perception  Laboratory  (VPL)  and  their  detection  decisions  analyzed.  Results  were 
analyzed  to  determine  which  uniforms  offered  the  most  protection  to  a  threat  sensor.  The 
laboratory  results  were  modeled  using  the  Fuzzy  Logic  Approach  (FLA)  with  a  resulting 
correlation  of  0.9. 


INTRODUCTION 

The  purpose  of  this  experiment  was  twofold.  The  primary  purpose  was  to 
determine  which  of  1 1  possible  uniforms  provided  the  most  protection  from  detection 
with  an  IR  imaging  sensor  to  a  soldier  at  night.  The  factors  that  were  varied  in  the 
experiment  were  the  range  from  the  soldier  to  the  sensor,  the  bandpass  of  IR  sensor, 
soldier  stance  and  the  type  of  uniform.  The  image  data  taken  in  the  field  was  converted 
and  transformed  for  display  on  a  computer  monitor  in  a  laboratory  setting.  The  secondary 
reason  to  perform  the  experiment  was  to  elucidate  the  concept  of  performing  visual 
detection  experiments  in  the  laboratory  setting  as  opposed  to  the  field.  Military  field  tests 
are  very  costly  because  of  the  security  and  equipment  needed  and  field  tests  also  have  a 
problem  with  control  of  the  important  variables,  such  as  ambient  lighting.  The  premise 
behind  a  visual  perception  laboratory  is  that  field  test  data  and  imagery  can  be 
supplemented  in  the  controlled  setting  of  the  laboratory  to  permit  a  more  careful  control 

of  the  viewing  conditions  and  permit  an  orderly  collection  of  data  from  a  large  number  of 
subjects. 
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EXPERIMENTAL  METHOD 


Participants 

A  total  of  37  active-duty  military  personnel  of  various  assignments  and  experience  levels 
participated.  All  the  participants  were  male  and  were  screened  to  verify  they  had  20-20 
(or  corrected  to  20-20)  vision.  The  participants  volunteered  for  the  study,  and  the  study 
was  conducted  during  normal  duty  shift  hours.  No  other  compensation  was  provided. 

Stimuli  and  Apparatus. 

The  stimuli  were  black  and  white  digital  IR  images  of  outdoor  scenes.  The  target  stimuli 
contained  an  image  ol  a  soldier  in  a  background,  while  the  distracter  images  presented  the 
empty  background.  The  combination  of  experimental  factors  dictated  a  total  of  216 
t‘u get-present  stimuli.  A  single  target-absent  stimulus  was  created  for  each  target-present 
factor  level  combination.  The  target-absent  stimuli  were  duplicated  to  balance  the 
pioportion  of  target-present  and  target-absent  images.  The  images  were  recorded  on  to 
VHS  tape  at  night  using  two  different  infrared  imaging  cameras,  shortwave  and 
longwave.  Examples  of  the  stimuli  images  are  shown  below  in  Fig.’s  1  and  2.  The 
imaging  sensor  specifications  are  as  shown  in  Table  I.  The  VHS  tape  was  digitized  using 
a  Cosmo  video  capture  board  on  a  Silicon  Graphics  Indigo  workstation.  The  images  were 
digitized  in  RGB  format  at  a  resolution  of  320  X  240  pixels.  These  images  were  then 
converted  to  a  Windows  95-compatible  format  for  presentation.  At  a  nominal  viewing 
distance  of  25  in,  the  entire  stimulus  subtended  a  visual  angle  of  8.64  degrees  in  the 
horizontal,  and  6.48  degrees  in  the  vertical,  while  the  instantaneous  field  of  view  was 
0.03  degrees  per  pixel.  The  stimulus  presentation  and  data  collection  routine  was 
programmed  in  Visual  Basic.  The  timing  of  image  presentation  and  response  data  was 
accomplished  with  millisecond  accuracy  by  calling  the  Windows  95  multimedia  library. 
The  Visual  Basic  program  also  contained  the  necessary  routines  for  interactively 
controlling  the  pace  and  progression  of  the  experiment,  as  well  as  providing  initial 
instructions  and  data  logging.  The  keyboard  was  used  to  log  response  information. 


Fig.  1  Standing  soldier 


Fig.  2  Crouched  soldier 


TABLE  I  Field  sensor  properties 


Parameter 

Shortwave 

Longwave 

bandpass  (pm) 

2.75  to  4.50 

8.0  to  12.0 

IFOV  (deg) 

3.5  X  2.8 

2.0  X  1.3 

hor  IFOV  (mrad) 

0.119 

0.1135 

ver  IFOV  (mrad) 

0.1007 

0.0675 

Procedure 

Observers  participated  in  three  sessions.  Each  session  had  an  identical  format.  The 
sessions  started  with  an  initialization  screen  where  the  experimenter  entered  the 
participant  s  identification  information.  This  was  followed  with  a  screen  on  instructions. 
A  brief  background  on  the  task  situation  was  provided  and  the  participants  were 
instructed  that  some  scenes  contained  soldiers  and  others  did  not.  Further,  based  on  their 
perception  of  the  scene,  they  were  given  three  response  options:  ( 1 )  yes  was  reserved  for 
cases  where  the  participant  was  sure  they  saw  a  soldier  in  the  field;  (2)  maybe  was  used  to 
indicate  that  the  participants  saw  something  suspicious,  but  couldn’t  be  sure  it  was  a 
soldier;  (3)  no  was  reserved  for  cases  where  the  participant  was  sure  there  was  not  a 
soldier  in  the  scene. 

After  viewing  instructions  the  participant  hit  the  return  key  to  continue,  and 
started  the  first  trial  sequence.  Each  trial  sequence  proceeded  in  the  following  manner: 
First,  an  alert  box  popped  up  on  the  screen  telling  the  participant  to  hit  the  space  bar  to 
continue.  After  hitting  the  space  bar  a  mask  screen  appeared  with  a  noise  pattern.  The 
noise  pattern  remained  on  the  screen  for  an  interval  varying  uniformly  from  750ms  to 
2000ms  to  prevent  anticipation  effects  in  the  pattern  of  responses.  After  the  initial  noise 


interval,  the  stimulus  image  appeared  on  the  screen  for  7  seconds.  The  participant  was 
allowed  to  respond  at  any  point  after  the  stimulus  image  appeared.  If  the  participant  did 
not  respond  within  7  seconds,  a  second  noise  mask  appeared  in  place  of  the  stimulus 
image,  with  an  additional  prompt  for  the  participant  to  respond.  The  Z,  V,  and  M  keys 
were  labeled  for  use  as  the  yes,  maybe,  and  no  response  keys,  respectively.  This  trial 
sequence  proceeded  throughout  the  randomly  ordered  trials  dictated  by  the  experimental 
design  for  all  three  sessions. 

Design  and  Treatment  of  Data 

Each  of  the  three  sessions  served  as  a  replicate  in  a  full-factorial  experimental  design. 
Stimulus  presentation  order  was  randomized  separately  for  each  session.  The 
independent  variables  were  Sensor  (short  wave,  long  wave).  Posture  (prone,  crouched, 
standing).  Range  (4  levels,  ranging  from  target  sizes  of  approximately  5  degrees  to  10 
minutes  of  arc),  and  Uniform  (9  different  uniforms).  There  was  a  no-target  image  for 
each  target-present  image  in  the  experimental  design.  The  resulting  yes,  no,  and  maybe 
responses  were  classified  into  hits,  false  alarms,  misses,  and  correct  rejections.  This 
information  was  used  to  develop  estimates  of  observer  sensitivity  (d)  at  two  different 
response  criterion  levels,  using  the  rating  scale  method  shown  in  McMillan  and  Creelman 
(1996).  The  estimated  true  d  was  calculated  as  the  mean  of  the  liberal  and  conservative  d 
values.  ANOVA  was  performed  on  mean  d  value,  calculated  across  all  observers  within 
each  experimental  cell. 


ANALYSIS 

The  graph  on  the  left  side  in  Fig.  1  shows  the  value  of  the  probability  of  saying 
yes  given  that  a  target  is  in  the  picture,  P(Y)  versus  sensor  and  uniform.  The  P(Y)  was 
computed  on  the  basis  of  a  YES  response  (Y)  and  the  graph  on  the  right  side  in  Fig  1  was 
prepared  using  the  YES  and  UNCERTAIN  (Y+?)  response  as  the  basis  for  the 
computation  of  the  d-prime  and  probability  of  saying  yes,  or  since  the  task  was  detection, 
probability  of  detection  (Pd).  The  YES  (Y)  response  gives  a  high  criterion  for  a  scene  to 
contain  a  target.  The  (Y+?)  response  aggregation  is  a  little  more  forgiving  and  basically 
provides  a  Receiver-Operator-Characteristic  (ROC)  curve  that  gives  a  medium  criterion 
for  selecting  a  scene  as  containing  a  target.  Each  plot  also  shows  how  P(Y)  changed 
versus  run  number.  There  were  3  runs  per  subject  and  because  of  time  constraints,  the 
soldiers  had  to  do  the  tests  sequentially  with  no  break,  so  their  performance  did  degrade 
slightly.  For  both  sensors,  system  8  is  the  system  that  provides  the  best  camouflage  to  the 
soldier  in  the  backgrounds  used  in  the  field  data  collection.  Sensor  2  was  the  short  wave 
(3-5  micron)  sensor  and  sensor  1  was  the  longwave  (8-12  micron)  system.  Generally,  the 
shortwave  (SW)  systems  have  a  higher  resolution,  or  have  greater  contrast,  than  the 
longwave  (LW)  systems.  However,  the  LW  systems  generally  perform  better  under 
adverse  weather  conditions  and  for  long  ranges  because  of  the  wider  window  between  8 
and  1 2  microns  and  less  energy  loss  due  to  atmospheric  scattering.  It  therefore  makes 
sense  that  the  LW  system  performed  better  over  range  and  other  environmental  variables 
which  caused  the  various  uniform  systems  to  be  more  detectable  through  the  LW  IR 
sensor. 


Effect  of  uniform 
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Fig.  1  Comparison  of  the  Probability  of  detection  by  uniform  and  sensor 


For  each  sensor,  there  is  a  monotonic  decrease  in  dprime  as  a  function  of  distance 
Fig.  2  shows  how  the  detectability  metric  d-prime  changed  as  a  function  of  distance  for 
both  SW  and  LW  systems.  In  this  graph  d-prime  was  aggregated  over  all  factors  and  boll 
response  bias’s  are  shown.  The  curves  of  the  detectability  decrease  with  distance.  An 
exponential  Fit  to  the  curve  explains  greater  than  95%  of  the  variance.  The  rate  of  descen 


of  d-prime  versus  distance  is  a  little  steeper  for  sensor  2  than  sensor  1 ,  which  makes  sens< 
because  the  performance  of  the  SW  system  is  known  to  decrease  very  fast  with  increasing 
range.  SW  systems  are  best  used  for  very  close-up,  short-range  surveillance.  Sensor  2 
was  the  short  wave  (3-5  micron)  sensor  and  sensor  1  was  the  longwave  (8-12  micron) 


system.  Generally,  the  shortwave  (SW)  systems  have  a  higher  resolution,  or  have  greater 
contrast,  than  the  longwave  (LW)  systems.  As  shown  above  in  slide  2,  the  LW  system 


all  the  uniforms. 


Fig.  2  Effect  of  distance  on  dprime 


Fig.  3  below  shows  that  the  higher  the  soldier  was  relative  to  the  ground,  the  greater  was 
his  chance  of  being  detected.  Again,  this  result  agrees  with  common  sense  and  is  not 
surprising. 


Fig.  4  shows  how  dprime  aggregated  over  all  parameters  varied  as  the  run  number 
increased.  It  is  quite  surprising,  actually  how  little  the  performance  degraded.  Fig.  5 
plots  the  time  to  respond  versus  run  number  and  illustrates  the  effect  of  learning. 
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Fig.  4  Dprime  versus  run 
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Fig.5  Time  to  answer  versus  run 


Aside:  d’  is  a  standard  pyschophysical  measure  of  perceptibility.  It  is  computed  from  raw 
test  data  as  the  mean  z-score  for  correct  detections  minus  the  z-score  of  false  alarms, 
normalized  to  the  standard  deviation  in  the  z-score.  It  is  a  measure  of  the  visual  signal 
normalized  to  the  visual  noise,  i.e.,  it  is  the  number  of  standard  deviations  from  the  mean 

d'  is  easily  related  to  our  sense  of  the  ease  or  difficulty  of  detection 
d’  =  0  =>  not  detectable 
d’  =  1  =>  hard  to  detect 

d’  =  2  =>  variable  detection  response  among  subjects  or  at  different  times 

d’  =  3  =>  easy  to  detect 

d’  =  4  =>  pop-out  target  detected  instantly 

Computationally  for  this  experiment,  we  used  the  following  from  equations  (1)  below 
Pd=\-0(k-cF) 

pfa=\-m) 

where, 

O  is  the  left  tail  of  the  normal  distribution 
d’  is  the  target  detectability 
k  is  the  particular  criterion  . 

The  dprime  of  each  uniform  as  seen  through  the  two  threat  sensors  is  plotted  against  each 
other  to  see  which  uniform  performed  the  best  under  all  circumstances  and  is  shown 
below  in  Fig.  6.  System  8  is  the  hardest  to  detect,  and  system  1  is  the  easiest  to  detect  for 
both  sensors.  Hence,  system  8  is  the  system  to  wear  if  you  want  to  avoid  detection  in  the 
field. 
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Hg.  6  C  omparison  of  uniform  performance  by  sensor 


The  graph  in  Fig.  7  indicates  that  system  8,  12  and  13  took  the  longest  time  to  detect, 
which  coincides  perfectly  with  the  observation  from  previous  plots  that  system  8  was  the 
hardest  to  detect.  As  time  progressed,  and  the  experiment  proceeded  to  run  2  and  run  3. 
This  decreased  reaction  time  can  be  explained  by  learning  on  the  part  of  soldiers. 


Effect  of  uniform 
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The  relative  importance  of  the  factors  considered  in  the  experiment  was 
determined  by  the  ANOVA.  Distance,  posture,  sensor  type,  uniform,  and  uniform 
crossed  with  sensor  were  equally  important  and  the  most  important  factors.  In  decreasing 
order  of  importance  were  the  following,  uniform  crossed  with  distance,  sensor  crossed 
with  posture,  posture  crossed  with  distance,  uniform  crossed  with  posture,  sensor  crossed 
w'ith  distance,  and  lastly  run  number.  The  fact  that  run  number  is  the  lest  significant 
factor  is  reassuring  since  there  was  some  concern  about  the  attention  degradation  of  the 
subjects.  Finally,  the  statistical  model  explains  80%  of  the  variance  in  the  data. 


Table  11 

Analysis  of  Variance 


Source 

P 

DISTANCE 

0.000000 

POSTURE 

0.000000 

RUN 

0.005394 

SENSOR 

0.000000 

UNIFORM 

0.000000 

POSTURE*DlSTA 

NCE 

0.000614 

SENSOR*UNIFOR 

M 

0.000000 

POSTURE*  UNIFO 
RM 

0.000214 

DISTANCE*SENS 

OR 

0.002407 

SENSOR*POSTUR 

E 

0.000325 

UNIFORM*DISTA 

NCE 

0.000043 

Note:  The  P-value  is  the  probability  that  the  test  statistic  will  take  on  a  value  that  is  at 
least  as  extreme  as  the  observed  value  of  the  statistic  when  the  null  hypothesis  is  true,  or, 
the  P-value  is  the  smallest  level  of  significance  that  would  lead  to  rejection  of  the  null 
hypothesis,  from  Montgomery’s  Design  and  Analysis  of  Experiments. 


Fuzzy  Logic  (Multivalued  Logic  )  Approach  (FLA)  to  modeling  the  data 


The  theory  behind  the  computation  of  target  detection  probabilities  in  the  thermal  and 
visible  parts  of  the  electromagnetic  spectrum  has  been  discussed  in  4'5'6.  The  theory  of  the 
fuzzy  logic  approach  (FLA)  and  the  application  of  the  FLA  to  the  problem  of  computing 
target  acquisition  probabilities  to  targets  in  both  static  infrared  and  visual  scenes  has  been 
described  in  other  papers  7,8<\  The  theory  remains  the  same  in  this  paper.  The  novel 
application  of  the  FLA  in  this  research  was  the  inclusion  of  uniform,  distance  and  posture 
as  fuzzy  logic  categories.  A  picture  of  the  FIS  used  to  analyze  and  model  the  data  from 
this  experiment  is  shown  below  in  Fig.  1 1. 


Fig.  8  Fuzzy  Inference  System  (FIS)  for  used  for  modeling  the  data 

Fig.  9  through  Fig.  12  are  graphs  of  the  experimental  results  versus  the  FLA  predictions. 
Fig.  9  shows  a  graph  of  the  FLA  predicted  Pd  and  experimental  Pd  versus  image  number 
for  one  sensor  and  soldier  posture.  In  this  selection  of  variables  the  correlation  between 
FLA  model  and  experimental  results  was  0.84. 


Fig.  9  Laboratory  data  and  FLA  predictions 


Fig.  10  Laboratory  data  and  FLA  predictions 


Fig.  11  Laboratory  data  and  FLA  predictions 


CHAMELEON  PLOTS 


CAMAELEON  [1 1,12]  is  a  computer  model  that  emulates  human  early  vision  and 
computes  the  visual  detectability  of  a  vehicle  in  a  background.  CAMAELEON  takes  a 
TIFF  image  as  input  and  then  decomposes  it  into  a  set  of  feature  images.  The  model 
computes  the  histogram  overlap  between  user  defined  target  and  background  regions. 

The  target  detection  metric  is  computed  from  the  overlap  of  histograms  for  the  target  and 
background.  CAMAELEON  is  one  of  the  computational  early  vision  models  the  U.S. 
Army  is  evaluating  for  the  purpose  of  determining  the  visibility  of  ground  vehicles  under 
various  conditions  and  was  used  in  this  study  to  compare  against  laboratory  data  and 
fuzzy  logic  approach  (FLA)  predictions. 

The  figures  below  show  graphs  of  the  CAMAELEON  predictions  and  the 
perception  laboratory  results. 


P(y)*20  v&  Det  range 
correl:  0.86 
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APPENDIX  1 

>MODEL  DPRIME  =  CONSTANT  + 
DISTANCE+POSTURE+SENSOR+UNIFORM+POSTURE*. 
>DISTANCE+SENSOR*UNIFORM+POSTURE*UNIFORM+DISTANCE*SENSOR+ 
UNIFORM*DISTANCE+SENSOR*POSTURE  +  RUN 

Effects  coding  used  for  categorical  variables  in  model. 

Categorical  values  encountered  during  processing  are: 

DISTANCE  (4  levels) 

1.  2,  3,  4 

POSTURE  (3  levels) 

1,  2,  3 

SENSOR  (2  levels) 

1,  2 

UNIFORM  (9  levels) 

1.  2,  3,  4,  5,  6,  7, 

8,  9 

RUN  (3  levels) 

1,  2,  3 

Dep  Var:  DPRIME  N:  648  Multiple  R:  0.865319  Squared  multiple  R:  0.748777 
>CATEGORY  DISTANCE  POSTURE  RUN  SENSOR  UNIFORM  /  EFFECT 
>MODEL  DPRIME  =  CONSTANT  + 

DISTANCE+POSTURE+RUN+SENSOR+UNIFORM+POSTURE*, 

>DISTANCE+SENSOR*DISTANCE+UNIFORM*DISTANCE+SENSOR*POSTURE+ 

UNIFORM  *  POSTURE+, 

>UNLFORM*SENSOR 

>ESTLMATE 

Effects  coding  used  for  categorical  variables  in  model. 

Categorical  values  encountered  during  processing  are1 
DISTANCE  (4  levels) 

2,  3,  4,  5 

POSTURE  (3  levels) 

1,  2,  3 

RUN  (3  levels) 

1,  2,  3 

SENSOR  (2  levels) 

1,  2 


7, 


8, 


10, 


12, 


UNIFORM  (9  levels) 

1,  2,  5, 

13,  15 

3  case(s)  deleted  due  to  missing  data. 

Dep  Var:  DPRIME  N:  645  Multiple  R:  0.874878  Squared  multiple  R:  0.765412 


Analysis  of  Variance 


Source 

DISTANCE 

POSTURE 

RUN 

SENSOR 

UNIFORM 


POSTURE*DISTANCE 
SENSOR*DISTANCE 
UNIFORM*DISTANCE 
SENSOR*POSTUR£ 
UNIFORM  *POSTURE 
UNIFORM  *SENSOR 


472.186216 
406.316684 
10.518840  2 

227.155656 
515.055941 


Sum-of-Squares  df  Mean-Square  F-ratio  P 

3  157.395405  157.734389  0.000000 
2  203.158342  203.595885  0.000000 
5.259420  5.270747  0.005394 
1  227.155656  227.644883  0.000000 
8  64.381993  64.520653  0.000000 

6  3.996372  4.004979  0.000614 

3  4.846856  4.857295  0.002407 

24  2.633264  2.638936  0.000043 

2  8.127101  8.144604  0.000325 

16  2.801405  2.807438  0.000214 

8  5.825547  5.838094  0.000000 


23.978233 

14.540569 

63.198347 

16.254202 

44.822475 

46.604377 


Error 


567.777174  569  0.997851 


Durbin-Watson  D  Statistic  2.012 
First  Order  Autocorrelation  -0.006 


